Papers with finetuning method
Multi-dimensional Evaluation of Empathetic Dialogue Responses (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Prior efforts to measure conversational empathy focus on expressed communicative intents, but ignore the fact that conversation is also a collaboration involving both speakers and listeners. |
| Approach: | They propose a multi-dimensional empathy evaluation framework to measure both expressed intents from the speaker’s perspective and perceived empathy from the listener’s viewpoint. |
| Outcome: | The proposed framework measures both expressed intents from the speaker’s perspective and perceived empathy from the listener’s viewpoint. |
Pseudo Zero Pronoun Resolution Improves Zero Anaphora Resolution (2021.emnlp-main)
Copied to clipboard
| Challenge: | Masked language models have contributed to drastic performance improvements with regard to zero anaphora resolution (ZAR). |
| Approach: | They propose a pretraining task that trains MLMs on anaphoric relations with explicit supervision and a finetuning method that remedies a notorious discrepancy. |
| Outcome: | The proposed method improves zero anaphora resolution in Japanese ZAR . it uses a pretrain task and finetuning task to correct the discrepancy . |
Cross-lingual Social Misinformation Detector based on Hierarchical Mixture-of-Experts Adapter (2025.coling-main)
Copied to clipboard
| Challenge: | a global trend of misinformation is affecting non-native speaker users who are more susceptible to misinformation on foreign social media platforms. |
| Approach: | They propose a method to integrate sentiment analysis as an auxiliary task and a hierarchical routing strategy and expert-mask mechanism to enhance cross-lingual social misinformation detection. |
| Outcome: | The proposed method improves cross-lingual social misinformation detection in non-native speakers with only monolingual social media histories. |
Can Unsupervised Knowledge Transfer from Social Discussions Help Argument Mining? (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for argument mining are limited by the scarcity of manually annotated data and the highly domain-dependent nature of argumentation. |
| Approach: | They propose a novel transfer learning strategy to fine tune pretrained Transformer-based Language Models on a selectively masked language modeling task and a new prompt-based strategy for inter-component relation prediction. |
| Outcome: | The proposed method outperforms existing models on both within- and out-of-domain datasets while leveraging on the discourse context. |
A Unified Speaker Adaptation Approach for ASR (2021.emnlp-main)
Copied to clipboard
| Challenge: | Adapting a model to target speakers requires a lot of compute and may cause catastrophic forgetting to the existing speakers. |
| Approach: | They propose a unified speaker adaptation approach consisting of feature adaptation and model adaptation. |
| Outcome: | The proposed model outperforms baseline models with 20.58% relative WER reduction and surpasses finetuning method by 2.54% on target speaker adaptation. |
Controlled Low-Rank Adaptation with Subspace Regularization for Continued Training on Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to mitigate catastrophic forgetting can be broadly categorized into data-based, architecture-based and learning-based methods. |
| Approach: | They propose a subspace regularization method on LoRA structure that imposes constraints on direction of updating matrix’s null space. |
| Outcome: | The proposed method reduces scale of output change while introducing minimal constraint on model capacity. |
On the Relationship between Skill Neurons and Robustness in Prompt Tuning (2024.lrec-main)
Copied to clipboard
| Challenge: | Prompt Tuning is a parameter-efficient finetuning method for pre-trained large language models (PLMs). |
| Approach: | They propose to use RoBERTa to fine tune pre-trained large language models by finetuning only a small set of parameters to adjust for downstream tasks. |
| Outcome: | The proposed method activates specific neurons in the transformer’s feed-forward networks that are highly predictive and selective for the given task. |